Network control system, control apparatus, controlled apparatus, and apparatus control method

ABSTRACT

A network control system includes a control apparatus and a controlled apparatus. The control apparatus includes: a first execution unit which executes a first application program for operating the controlled apparatus; a display unit which displays an operation screen; an operational command generation unit which generates an operational command in response to an input by a user to the operation screen; and a first communication unit which outputs the operational command. The controlled apparatus includes: a second execution unit which executes a second application program, and identifies an operation type of the controlled apparatus; an operation type information generation unit; and a second communication unit which outputs the operation type information. The first execution unit generates an operation screen corresponding to the operation type, and the display unit displays the generated operation screen.

RELATED APPLICATIONS

This application is a Continuation of U.S. patent application Ser. No.13/981,580 filed on Jul. 24, 2013, now U.S. Pat. No. 9,348,427, which isthe U.S. National Phase under 35 U.S.C. § 371 of InternationalApplication No. PCT/JP2011/007282, filed on Dec. 26, 2011, which claimsthe benefit of Japanese Application No. 2011-014968, filed on Jan. 27,2011, the disclosures of which Applications are incorporated byreference herein.

TECHNICAL FIELD

The present invention relates to apparatuses connected to a network,such as digital home electronic apparatuses and mobile terminalstypified by smartphones, and particularly relates to a network controlsystem including a control apparatus which operates a controlledapparatus via a network.

BACKGROUND ART

Conventionally, digital home electronic apparatuses such as digitaltelevisions (hereinafter referred to as DTV) and HDD recorders areprovided with remote controllers dedicated to these apparatuses, andusers operate these apparatuses through the remote controllers.

Furthermore, in general, a remote controller has buttons (also referredto as “keys”), such as a channel button and a mute button, each assignedwith a function. With such a conventional remote controller, the userscan operate the apparatus very comfortably as long as he/she uses eachbutton only for the assigned function.

However, digital home electronic apparatuses available in recent yearsprovide keyword search function and Internet browsing function, wherebythe users have more opportunities to enter characters.

When the user enters characters using the conventional remote controllerhowever, he/she generally selects a target character by pressing down achannel number button and the like several times. In other words, withthe conventional remote controller, users cannot enter characters intodigital home electronic apparatuses in an intuitive manner but have toget used to the operation, which has been proven to be troublesome tothe users.

Accordingly, a technique has been disclosed in which a mobile terminalthat is suitable for entering characters, such as a mobile phone, isused as a remote controller for a digital home electronic apparatusinstead of the remote controller provided for the apparatus (forexample, see Patent Literature (PTL) 1). With the above technique, theuser can use an input unit such as a keyboard of the mobile terminalwhen entering characters, while using the mobile terminal for performingthe same operation as the remote controller provided for the apparatus.The above technique aims at improving the operability in the abovemanner.

CITATION LIST Patent Literature

[PTL 1] Japanese Unexamined Patent Application Publication No.2002-078042

SUMMARY OF INVENTION Technical Problem

However, with the conventional technique disclosed in PTL 1, the user isrequired to: determine whether or not the digital home electronicapparatus being operated is in a character input state; and when it isdetermined that the digital home electronic apparatus is in thecharacter input state, input characters using a keyboard.

In other words, the user has to understand the state of the digital homeelectronic apparatus to be operated, and select an optimal operationscheme. Accordingly, the conventional technique has a problem in thatthe user cannot select the optimal operation scheme well when, forexample, more than two operation schemes are available.

The present invention has been conceived in view of the above problem,and has an object to provide a network control system, a controlapparatus, a controlled apparatus, and an apparatus control method whichallows the control apparatus to operate the controlled apparatus moreefficiently.

Solution to Problem

A network control system according to an aspect of the present inventionis a network control system including a control apparatus and acontrolled apparatus which performs processing corresponding to anoperational command transmitted from the control apparatus via anetwork, wherein the control apparatus includes: a first execution unitconfigured to execute a first application program for operating thecontrolled apparatus; a display unit configured to display an operationscreen generated through the execution of the first application program;an operational command generation unit configured to generate theoperational command in response to an input by a user to the operationscreen displayed on the display unit; and a first communication unitconfigured to output the generated operational command to the network,the controlled apparatus includes: a second execution unit configured toexecute a second application program, and identify an operation typewhich is corresponding to processing performed by the second applicationprogram and is a type of operation accepted by the controlled apparatus;an operation type information generation unit configured to generateoperation type information indicating the identified operation type; anda second communication unit configured to output the generated operationtype information to the network, the first execution unit is configuredto receive the operation type information via the first communicationunit, and generate an operation screen corresponding to the operationtype indicated in the operation type information by executing the firstapplication program, and the display unit is configured to display thegenerated operation screen.

With this configuration, the controlled apparatus can identify theoperation type of the controlled apparatus, and notify the operationtype to the control apparatus. Furthermore, the control apparatus cangenerate and display an operation screen corresponding to the notifiedoperation type.

Thus, an operation screen suitable for the current operation type of thecontrolled apparatus is displayed on the control apparatus. As a result,the control apparatus can operate the controlled apparatus efficiently.

Furthermore, in the network control system according to an aspect of thepresent invention, the operation type information generation unit may beconfigured to generate the operation type information indicating atleast one of a keyboard type and a character string input scheme for usein the identified operation type, and the first execution unit may beconfigured to generate the operation screen corresponding to the atleast one of the keyboard type and the character string input schemeindicated in the operation type information.

With this configuration, the operation screen corresponding to the atleast one of the keyboard type and the character string input schemesuitable for operation of the controlled apparatus is displayed on thecontrol apparatus.

This allows, for example, automatically presenting to the user theoperation screen corresponding to the type of characters to betransmitted to the controlled apparatus. As a result, the controlapparatus can operate the controlled apparatus more efficiently.

Furthermore, in the network control system according to an aspect of thepresent invention, the second execution unit may be configured toidentify an operation type after the change when the operation type haschanged, the operation type information generation unit may beconfigured to generate operation type information indicating theoperation type after the change, the first execution unit may beconfigured to generate an operation screen corresponding to theoperation type after the change indicated in the operation typeinformation received via the first communication unit, and the displayunit may be configured to display the generated operation screencorresponding to the operation type after the change.

With this configuration, when the operation type of the controlledapparatus changes, an operation screen suitable for the operation typeafter the change is displayed on the control apparatus. In other words,even when the operation type of the controlled apparatus changes againand again, the controlled apparatus can switch between the operationscreens in response to the change.

Furthermore, in the network control system according to an aspect of thepresent invention, the operation type information generation unit may befurther configured to generate capacity information on processingcapacity of the controlled apparatus, the second communication unit maybe further configured to output the generated capacity information tothe network, and the first execution unit may be configured to receivethe capacity information via the first communication unit, and generatethe operation screen corresponding to the operation type indicated inthe operation type information and the processing capacity indicated inthe capacity information.

With this configuration, the capacity information indicating languagesthat can be interpreted by the controlled apparatus and so on istransmitted to the control apparatus via the network. Thus, the controlapparatus can generate and display an operation screen corresponding tothe operation type and the processing capacity of the controlledapparatus. As a result, the control apparatus can operate the controlledapparatus more efficiently.

Furthermore, in the network control system according to an aspect of thepresent invention, the operational command generation unit may beconfigured to generate the operational command corresponding to theinput from the user to the operation screen, the operational commandcomplying with Universal Plug and Play (UPnP) protocol.

With this configuration, UPnP protocol, which is a generic communicationprotocol, can be adopted as a communication protocol between the controlapparatus and the controlled apparatus. As a result, for example, itbecomes easier to establish the network control system.

Furthermore, in the network control system according to an aspect of thepresent invention, the control apparatus may further include a touch-padcontrol unit configured to obtain operation information identified fromgesture input from the user to the operation screen, and the operationalcommand generation unit may be configured to generate the operationalcommand corresponding to the operation information obtained by thetouch-pad control unit.

With this configuration, gesture input is allowed to the controlapparatus, which is the apparatus that the user actually operates,whereby the control apparatus can operate the controlled apparatus moreefficiently.

Furthermore, the control apparatus according to an aspect of the presentinvention can be realized as a control apparatus including acharacteristic constituent element of the control apparatus in thenetwork control system according to any one of the above aspects.

Furthermore, the controlled apparatus according to an aspect of thepresent invention can be realized as a controlled apparatus including acharacteristic constituent element of the controlled apparatus in thenetwork control system according to any one of the above aspects.

Furthermore, the apparatus control method according to an aspect of thepresent invention can be realized as an apparatus control methodincluding characteristic processing performed by the network controlsystem according to any one of the above aspects.

It is to be noted that the present invention can also be realized as aprogram which causes a computer to execute the apparatus control methodaccording to an aspect of the present invention. Furthermore, such aprogram can be distributed via a recording medium such as a CD-ROM or atransmission medium such as the Internet.

Moreover, the present invention can also be realized as an integratedcircuit including a characteristic constituent element of the controlapparatus according to an aspect of the present invention.

Moreover, the present invention can also be realized as an integratedcircuit including a characteristic constituent element of the controlledapparatus according to an aspect of the present invention.

Advantageous Effects of Invention

With the present invention, the user can always operate the controlledapparatus such as a digital home electronic apparatus by the optimalscheme, and can make full use of the digital home electronic apparatusmore intuitively. In other words, the control apparatus can operate thecontrolled apparatus efficiently.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 shows a schematic configuration of a network control systemaccording to an embodiment of the present invention.

FIG. 2 is a block diagram showing a primary configuration of a controlapparatus and a controlled apparatus according to the embodiment.

FIG. 3 is a block diagram showing a more specific configuration exampleof the control apparatus and the controlled apparatus according to theembodiment.

FIG. 4 is a flowchart showing an example of a flow of basic processingperformed by the network control system according to the embodiment.

FIG. 5 is a flowchart showing an example of apparatus control performedby the network control system according to the embodiment.

FIG. 6 shows a first example of transition to a different operationscreen in the control apparatus according to the embodiment.

FIG. 7 shows a second example of transition to a different operationscreen in the control apparatus according to the embodiment.

FIG. 8 shows a third example of transition to a different operationscreen in the control apparatus according to the embodiment.

FIG. 9 shows a fourth example of transition to a different operationscreen in the control apparatus according to the embodiment,

DESCRIPTION OF EMBODIMENTS Embodiment 1

An embodiment according to the present invention is described below withreference to Drawings. It is to be noted that each diagram is aschematic diagram, and thus is not necessarily an exact illustration.

Furthermore, the embodiment described below shows a preferable specificexample of the present invention. Numeric values, shapes, constituents,positions and topologies of the constituents, and the like in theembodiment are an example of the present invention, and it shouldtherefore not be construed that the present invention is limited to theembodiment. The present invention is determined only by the statement inClaims. Accordingly, out of the constituents in the followingembodiment, the constituents not stated in the independent claimsdescribing the broadest concept of the present invention are notnecessary for achieving the object of the present invention and aredescribed as constituents in a more preferable embodiment.

First, description is provided on a configuration of the network controlsystem in the embodiment, with reference to FIG. 1 to FIG. 3.

FIG. 1 shows a schematic configuration of the network control systemaccording to an embodiment of the present invention.

A network control system 10 in the embodiment includes a controlapparatus 200, a controlled apparatus 100, and a network 300 whichconnects the control apparatus 200 and the controlled apparatus 100.

The control apparatus 200 is, for example, a mobile terminal having atouch panel, such as a smartphone, a mobile phone, a remote controllerfor learning, a personal computer (PC), a portable video game player,and so on.

Furthermore, the controlled apparatus 100 is a digital home electronicapparatus, for example, a DTV, a hard disk drive (HDD) recorder, or aBlu-ray disc (BD) (registered trademark) player.

The controlled apparatus 100 is connected to the control apparatus 200via the network 300, and is controlled by the control apparatus 200.Specifically, the controlled apparatus 100 receives the operationalcommand transmitted from the control apparatus 200, and performsprocessing corresponding to the received operational command.

Furthermore, in the present embodiment, the controlled apparatus 100 cannotify the operation type of the controlled apparatus 100 (a type ofoperation accepted by the controlled apparatus 100) to the controlapparatus 200. Furthermore, the control apparatus 200 can display on thedisplay unit 221 an operation screen corresponding to the notifiedoperation type.

The network 300 connects a plurality of apparatuses one another via, forexample, a wireless local area network (LAN) router. It is to be notedthat wired communication may be used for a part or the entirecommunication route between the control apparatus 200 and the controlledapparatus 100.

FIG. 2 is a block diagram showing a primary configuration of the controlapparatus 200 and the controlled apparatus 100 according to theembodiment.

The control apparatus 200 includes a first execution unit 210, a displayunit 221, an operational command generation unit 230, and a firstcommunication unit 232.

The first execution unit 210 executes a first application program foroperating the controlled apparatus 100. The display unit 221 displays anoperation screen generated through the execution of the firstapplication program.

The operational command generation unit 230 generates the operationalcommand in response to an input by a user to the operation screendisplayed on the display unit 221. The first communication unit 232outputs the generated operational command to the network 300.

Furthermore, the controlled apparatus 100 includes a second executionunit 110, an operation type information generation unit 130, and asecond communication unit 132.

The second execution unit 110 executes a second application program, andidentifies an operation type which is corresponding to processingperformed by the second application program and is a type of operationto the controlled apparatus 100.

The operation type information generation unit 130 generates operationtype information indicating the identified operation type. The secondcommunication unit 132 outputs the generated operation type informationto the network 300.

The operation type information output from the controlled apparatus 100to the network 300 in the above manner is received by the controlapparatus 200.

Specifically, the first execution unit 210 of the control apparatus 200receives the operation type information via the first communication unit232, and generates an operation screen corresponding to the operationtype indicated in the operation type information by executing the firstapplication program. The display unit 221 displays the generatedoperation screen corresponding to the operation type.

Here, the first application program is an operation program for thecontrol apparatus 200 to operate the controlled apparatus 100.Furthermore, the second application program is an application programfor providing the user with various services (display of video,communication with other apparatuses, and so on) by the controlledapparatus 100.

Described below are more specific configuration and processing of thecontrol apparatus 200 and the controlled apparatus 100, when the controlapparatus 200 is a mobile terminal such as a smartphone and thecontrolled apparatus 100 is a DTV.

FIG. 3 is a block diagram showing a specific configuration example ofthe control apparatus 200 and the controlled apparatus 100 according tothe embodiment.

The second execution unit 100 of the controlled apparatus 100 executesapplication programs (herein after referred to as “applications”) suchas a broadcast viewing application 101 for viewing broadcast programs, abrowser application 102 for allowing the user to view web pages etc. onthe Internet, and so on. The broadcast viewing application 101 and thebrowser application 102 are an example of the second applicationprogram.

Specifically, each of the applications is executed via a graphical userinterface (GUI) control unit 111 of the second execution unit 110. Eachapplication receives, from the GUI control unit 111, operationinformation described later. Furthermore, each application outputs GUIscreen information in order to request the GUI control unit 111 to drawa GUI screen.

Upon receipt of the GUI screen information from the application, the GUIcontrol unit 111 converts the received GUI screen information into avideo signal, and transmits the signal to the display unit 121.Furthermore, the GUI control unit 111 identifies the operation type ofthe controlled apparatus 100 from the received GUI screen information.In other words, an operation type corresponding to the processingperformed by the application is identified. A specific example of theoperation type is described later using FIG. 6 and so on.

The display unit 121 displays the received video signal on the display.Furthermore, the video signal output from the GUI control unit 111 isalso output to a video signal terminal (not shown).

It is to be noted that, when the controlled apparatus 100 is anapparatus which does not have a display apparatus for displaying video,such as an HDD recorder, the display unit 121 can be realized with anexternal display apparatus connected to a video terminal of thecontrolled apparatus 100.

In the present embodiment, the controlled apparatus 100 includes adedicated remote controller (not shown) which communicates usinginfrared (IR) for example, and the user basically operates thecontrolled apparatus 100 through the dedicated remote controller.

Operation information of the dedicated remote controller (for example,pressing of a channel number button and a menu button) is received at anIR remote controller control unit 122, and the received operationinformation is notified to an input control unit 112.

The input control unit 112 is an element which gathers operationinformation on the controlled apparatus 100. For example, when a mouse(not shown) is connected, the input control unit 112 activates a mousecontrol unit 123 to receive operation information from the mouse. Theinput control unit 112 notifies the gathered operation information tothe GUI control unit 111.

The GUI control unit 111 notifies the notified operation information tothe application currently being executed. For example, when thebroadcast viewing application 101 is being executed and a channel-upsignal is being transmitted from the dedicated remote controller, thebroadcast viewing application 101 changes the channel. In this manner,the controlled apparatus 100 executes an operation corresponding to theoperation from the user.

In addition to the above processing, the controlled apparatus 100 in thepresent embodiment performs the following processing by the secondcommunication unit 132 and a protocol processing unit 131. Specifically,the second communication unit 132 controls the network communication.Furthermore, the protocol processing unit 131 detects an apparatus onthe internet 300 and processes an operational command from the controlapparatus 200. Moreover, the operation type information generation unit130 of the protocol processing unit 131 generates operation typeinformation indicating the operation type identified by the GUI controlunit 111.

For example, when the network 300 uses Ethernet (registered trademark),the second communication unit 132 corresponds to TCP/IP communicationsprocessing stack. Furthermore, the protocol processing unit 131corresponds to, for example, Universal Plug and Play (UPnP) protocolstack. The protocol processing unit 131 replies to the apparatusdetection command which is described later from the control apparatus200, and performs processing to analyze the operational command from thecontrol apparatus 200. A specific example of the processing performed bythe protocol processing unit 131 is described later using FIG. 5 and soon.

The control apparatus 200 has almost the same configuration as thecontrolled apparatus 100, for the communication via the network 300. Thecontrol apparatus 200 executes a network remote control application 201etc. for operating the controlled apparatus 100. The network remotecontrol application 201 is an example of the first application program.

Since a type of processing performed by the GUI control unit 211, thedisplay unit 221, the input control unit 212, and the firstcommunication unit 232 is the same as that of their respectivecounterparts in the controlled apparatus 100, the description isomitted.

The control apparatus 200 differs from the controlled apparatus 100 inthat the operation information identified from the gesture input made bythe user to the operation screen is obtained by a touch-pad control unit222 connected to the touch panel. In other words, the operationalcommand generation unit 230 generates an operational commandcorresponding to the operation information obtained by the touch-padcontrol unit 222.

It is to be noted that the touch panel is located on a front surface ofa panel, such as a liquid crystal display panel, of the display unit221.

Furthermore, the protocol processing unit 231 is a counterpart of theprotocol processing unit 131 provided for the controlled apparatus 100.Specifically, the protocol processing unit 231 detects the controlledapparatus 100 by transmitting an apparatus detection command.

Furthermore, the operational command generation unit 230 of the protocolprocessing unit 231 generates an operational command for controlling thecontrolled apparatus 100 in response to an operation performed on thenetwork remote control application 201. The generated operationalcommand is transmitted to the controlled apparatus 100 via the firstcommunication unit 232.

Next, an operation performed by the network control system 10 in theembodiment is described with reference to FIG. 4 to FIG. 9.

FIG. 4 is a flowchart showing an example of a flow of basic processingperformed by the network control system 10 according to the embodiment.It is to be noted that, in FIG. 4, the “DTV” corresponds to thecontrolled apparatus 100 and the “mobile terminal” corresponds to thecontrol apparatus 200. This goes the same for FIG. 5 described later.

In the network control system 10 according to the embodiment, the secondexecution unit 110 of the controlled apparatus 100 identifies thecurrent operation type (S1).

The operation type information generation unit 130 of the controlledapparatus 100 generates operation type information indicating theidentified operation type (S2).

The second communication unit 132 transmits the generated operation typeinformation to the control apparatus 200 via the network 300 (S3).

The first communication unit 232 of the control apparatus 200 receivesthe operation type information (S4). The received operation typeinformation is transmitted to the first execution unit 210, and anoperation screen corresponding to the operation type indicated in theoperation type information is generated. The generated operation screenis displayed on the display unit 221 (S5).

The following describes a specific behavior of the network controlsystem 10 which performs the above basic processing using the flowchartshown in FIG. 5, and FIG. 6 which shows an example of the transition toa different operation screen displayed by the control apparatus 200.

First, when the controlled apparatus 100 and the control apparatus 200are connected to the network 300, the second communication unit 132 andthe first communication unit 232 complete preparation for communicationby obtaining the IP address (S11).

When the above communication units are ready to communicate, theprotocol processing unit 231 of the control apparatus 200 transmits anapparatus detection command. When no reply for the apparatus detectioncommand is received (No in S12) the protocol processing unit 231 keepstransmitting the apparatus detection command periodically until thecontrolled apparatus 100 is detected.

When the controlled apparatus 100 receives the apparatus detectioncommand, the protocol processing unit 131 transmits a reply packet.Thus, the control apparatus 200 detects the controlled apparatus 100(Yes in S12).

The control apparatus 200 can obtain, by detecting the controlledapparatus 100, the capacity information and operation type informationof the controlled apparatus 100.

The first execution unit 210 displays the optimal control operationscreen based on the above information by executing the network remotecontrol application 201, on the display unit 221 of the controlapparatus 200 (S13).

Here, the capacity information is information on the processing capacityof the controlled apparatus 100, and is, for example, information onlanguages displayed and interpreted by the controlled apparatus 100, andon applications operable on the controlled apparatus 100. The capacityinformation is generated by, for example, the operation type informationgeneration unit 130.

For example, when the controlled apparatus 100 can interpret English,capacity information indicating the fact is generated, and transmittedto the control apparatus 200. Upon receipt of the capacity information,the first execution unit 210 can cause the display unit 221 to displaythe description, name of the button, and so on in English, by executingthe network remote control application 201.

Furthermore, for example, when the controlled apparatus 100 hasthree-dimensional (3D) display capacity, capacity information indicatingthe fact is generated, and transmitted to the control apparatus 200.Upon receipt of the capacity information, the first execution unit 210can cause the display unit 221 to display a 3D setting button for 3Ddisplay setting of the controlled apparatus 100.

As described above, the operation type information is information on thetype of operation accepted by the controlled apparatus 100, and anexample of the operation type information includes the following variouspieces of information:

(1) Information indicating that the controlled apparatus 100 acceptsbasic operation and is in a state to accept various commands eachcorresponding to each key of the IR remote controller provided;

(2) Information indicating that the controlled apparatus 100 is in astate to accept character strings;

(3) Information indicating that the controlled apparatus 100 is in astate to be operable with a mouse and is in a state to accept vectorinformation;

(4) Information indicating that the controlled apparatus 100 is in astate to be operable with audio input and is in a state to accept audioinformation;

(5) Information indicating that the controlled apparatus 100 is in astate to accept gesture input or gyro operation; and

(6) Information indicating that the controlled apparatus 100 is in astate to accept gamepad input.

As described above, the operation type information in the presentembodiment is, more specifically, information identifying a type ofoperation accepted by the controlled apparatus 100 that is non-imageinformation. In other words, the controlled apparatus 100 transmits tothe control apparatus 200 not the operation screen itself but theoperation type information indicating what type of operation can beaccepted (that is the current operation type). Upon receipt of suchoperation type information, the control apparatus 200 generates anddisplays an operation screen corresponding to the operation type. Thus,the control apparatus 200 can control the controlled apparatus 100 moreefficiently.

It is to be noted that each of the above (1) to (6) is an example of theoperation type information, and the operation type information is notlimited to the above information (1) to (6).

Furthermore, these operation types can be designated concurrently, andthe first execution unit 210 may cause the display unit 221 to displayan operation screen which satisfies all the operation types designatedconcurrently. Furthermore, the first execution unit 210 may select oneof the operation types designated concurrently based on the prioritiesand the like, and cause the display unit 221 to display the optimaloperation screen for the selected operation type.

Here, a case is assumed where the controlled apparatus 100 is in a stateto accept the basic operation. In this case, the control apparatus 200draws the basic operation screen 260 shown in (a) in FIG. 6 on thedisplay unit 221 provided with a touch panel on the front surface.

For example, when the controlled apparatus 100 is executing thebroadcast viewing application 101, the controlled apparatus 100transmits: the operation type information indicating the above (1); andthe capacity information indicating that the controlled apparatus 100 isdisplaying in Japanese, to the control apparatus 200 by the secondcommunication unit 132.

Thus, the basic operation screen 260 shown in (a) in FIG. 6 for exampleis displayed on the display unit 221 of the control apparatus 200. Inthe basic operation screen 260, for example the user can transmit to thecontrolled apparatus 100 an instruction for volume control, channelswitch, and so on, with gesture input. The gesture input is an inputscheme by which the user performs gesture operation on the operationscreen for inputting an instruction corresponding to the gesture to anapparatus.

Such gesture operation includes: swipe operation to swipe the surface ofthe touch panel (display unit 221) with one or more fingers; or tapoperation to tap the surface one or more times with a finger.

When the user performs the above-described operation on the operationscreen displayed on the display unit 221 (Yes in S14), the operationalcommand generation unit 230 of the protocol processing unit 231generates an operational command corresponding to the operation. Thegenerated operational command is transmitted to the controlled apparatus100 via the first communication unit 232 (S15).

Here, as an example, it is assumed that the user has performed a gestureoperation to swipe the touch panel in the upper direction. In this case,the control apparatus 200 transmits, as an operational command, anoperational command including an IR remote control code that correspondsto the channel-up to the controlled apparatus 100 via the network 300.

In the controlled apparatus 100, when the second communication unit 131receives the above operational command, the protocol processing unit 131interprets the content of the operational command. The protocolprocessing unit 131 notifies the interpreted content, via the inputcontrol unit 112 and the GUI control unit 111, to the application beingexecuted (here, the broadcast viewing application 101). The broadcastviewing application 101 changes the channel to the next higher channel,as processing corresponding to the operational command (S16).

It is to be noted that, in the basic operation screen 260 shown in (a)in FIG. 6, various operational commands are associated with variousgesture operation. However, the control apparatus 200 does notnecessarily have to support information input by gesture operation(gesture input). For example, the control apparatus 200 may be providedwith a channel change button for controlling channel-up andchannel-down.

Furthermore, the type of the gesture operation associated with theoperational command by the control apparatus 200 is not specificallylimited. For example, an operational command indicating channel-up maybe transmitted to the controlled apparatus 100, when the touch panel isswiped with two fingers in the upper direction.

Furthermore, for example, when the touch panel is swiped with twofingers in the lower direction, an operational command indicatingchannel-down may be transmitted to the controlled apparatus 100.

Furthermore, for example, when the touch panel is swiped with twofingers to the right, an operational command indicating input switchingmay be transmitted, and when the touch panel is swiped with two fingersto the left, an operational command indicating broadcast wave switchingmay be transmitted, to the controlled apparatus 100.

Moreover, another example for associating the gesture operation and theoperational command includes: in the case where the user touches thetouch panel with two or more fingers, volume up when the user moves thefingers to draw a circle to the right, while volume down when the usermoves the fingers to draw a circle to the left; and mute when the userpinched the touch panel with two fingers (performed pitch operation).

Furthermore, a button may be displayed to turn off the controlledapparatus 100 by shuffling the control apparatus 200 (shaking thecontrol apparatus 200). Furthermore, the control apparatus 200 maytransmit an operational command instructing turning off the controlledapparatus 100 when the button to turn off is pressed, and concurrentlyterminate the network remote control application 201 operating on thecontrol apparatus 200.

After the processing or when no user operation is performed on theoperation screen (No in S14), the control apparatus 200 checks whether anotification on a change in the operation type is received from thecontrolled apparatus 100 (S17). In other words, the control apparatus200 checks whether new operation type information is received. When nonew operation type information is received (No in S17), the controlapparatus 200 proceeds to processing of S14 and waits for an operationfrom the user.

Here, as an example, it is assumed that a browser application 102 isactivated while the controlled apparatus 100 is in a broadcast viewingstate.

For example, a case is assumed in which, in a state shown in (a) in FIG.6, the browser application 102 is activated as shown in (b) FIG. 6, anda certain web page is displayed on the display unit 121 of thecontrolled apparatus 100.

In this case, the operation type of the controlled apparatus 100 ischanged, and the second execution unit 110 identifies the operation typeafter the change (Yes in S17).

Specifically, it is identified that a vector input (browser operation)can be accepted as the operation type after the change, and theidentified operation type is notified from the controlled apparatus 100to the control apparatus 200.

In other words, operation type information indicating that a cursor(also referred to as a pointer) is in a movable state is generated asinformation indicating the operation type after the change. Thegenerated operation type information is transmitted from the controlledapparatus 100 to the control apparatus 200 (S18).

Upon receipt of the operation type information, in the control apparatus200, the first execution unit 210 causes the display unit 221 todisplay, for example, the touch-pad mode screen 261 shown in (b) in FIG.6 (S13). In other words, an operation screen corresponding to theoperation type after the change is generated by the first execution unit210.

In the touch-pad mode screen 261, an operational command is generated bythe gesture operation performed on the touch-pad (realized by a touchpanel provided on the front surface of the display unit 221, in thepresent embodiment).

Specifically, operational commands each corresponding to drag operation,tap operation, pinch operation, and so on performed on the touch-pad istransmitted from the control apparatus 200 to the controlled apparatus100. In other words, in the touch-pad mode screen 261, various gestureinputs corresponding to various operational commands can be performed,in the same manner as in the basic operation screen 260.

In addition, as shown in (b) in FIG. 6, a scroll bar 261 a may bedisplayed on the touch-pad mode screen 261. This can also make itpossible to transmit a page scroll command to the controlled apparatus100.

Furthermore, a browser activation button 261 b may be displayed on thetouch-pad mode screen 261. When this browser activation button 261 b isselected by the user, the control apparatus 200 obtains from thecontrolled apparatus 100 a uniform resource locator (URL) of the webpage currently being displayed by the controlled apparatus 100, forexample.

Moreover, the control apparatus 200 accesses the obtained URL, anddisplays the web page corresponding to the URL on the browser of thecontrol apparatus 200. It is to be noted that when the URL indicates nota web page but multimedia content, the control apparatus 200 mayactivate a player corresponding to the multimedia content.

Here, a case is assumed in which, in response to the above-describedoperation performed by the user on the touch-pad mode screen 261, acursor is in the character input filed 150 of the web page as shown in(c) in FIG. 6. It is to be noted that the cursor in the character inputfiled 150 is also described as a caret, for example.

In this case, the controlled apparatus 100 is in a state waiting forinput of English character strings for example (Yes in S17), andtransmits to the control apparatus 200 operation type informationindicating the fact (S18).

With the transmission of the operation type information, the controlapparatus 200 receives the notification of the operation type after thechange, proceeds to processing of S13, and displays an operation screensuitable for the notified operation type.

Here, since the input waiting state for English character strings isnotified, the control apparatus 200 displays, for example as shown in(c) in FIG. 6, the English keyboard screen 262 on the touch panel.

More specifically, the first execution unit 210 causes the display unit221 to display the English keyboard screen 262 that is the operationscreen corresponding to the operation type.

In other words, the operation screen displayed on the control apparatus200 is automatically switched in response to the change in the operationtype of the controlled apparatus 100. Thus, the user can easily knowthat the controlled apparatus 100 is waiting for an input of Englishcharacter strings. Moreover, the user can input character strings to thecontrolled apparatus 100 with the control apparatus 200 immediately,using the touch panel (display unit 221), a hardware keyboard (notshown), and so on, in the way he/she is used to.

Here, regarding the character string, when the controlled apparatus 100is waiting not for English character strings but for number characterstrings, operation type information including information indicating thefact is transmitted from the controlled apparatus 100 to the controlapparatus 200.

For example, as shown in (a) in FIG. 7, a case is assumed in which thecursor is in the character input field 151 and the controlled apparatus100 is waiting for an input of an English character string. In thiscase, in the same manner as in (c) in FIG. 6, the English keyboardscreen 262 is displayed on the control apparatus 200.

Furthermore, the character string displayed in the display field 250,which is for displaying the input character on the English keyboardscreen 262, is transmitted to the controlled apparatus 100 by apredetermined operation and is input to the character input field 151.

It is to be noted that how to transmit one or more characters from thecontrol apparatus 200 to the controlled apparatus 100 is described laterwith reference to FIG. 8 and FIG. 9.

Assume that, for example as shown in (b) in FIG. 7, the cursor is movedto the number input field 152 to which numbers such as a postal code isdesired to be input.

In this case, the operation type information indicating that thecontrolled apparatus 100 is waiting for an input of a number characterstring is transmitted from the controlled apparatus 100 to the controlapparatus 200.

As a result, the control apparatus 200 displays a number keyboard screen263 as shown in (b) in FIG. 7. Thus, the user can input numbersimmediately.

Furthermore, as described above, the controlled apparatus 100 caninclude, in the operation type information, information other than thekeyboard type which is information indicating the controlled apparatus100 is waiting for input of what type of character string, and notifythe information to the control apparatus 200.

For example, regarding the character string input scheme, the controlledapparatus 100 may notify the control apparatus 200 of operation typeinformation indicating whether the controlled apparatus 100 preferscharacters to be input character by character or characters bycharacters.

As an example, a case is assumed in which the controlled apparatus 100transmits to the control apparatus 200 information indicating thatcharacters should be input characters by characters, as informationindicating the character string input scheme. In this case, in thecontrol apparatus 200, a transmission button 251 displaying “GO” isdisplayed on the English keyboard screen 262 as shown in (a) in FIG. 8.Specifically, the first execution unit 210 which has received the inputscheme information causes the display unit 221 to display the Englishkeyboard screen 262 including the transmission button 251.

In this case, the characters input to the control apparatus 200 usingthe English keyboard screen 262 and then displayed in the display field250 are simultaneously transmitted to the controlled apparatus 100through the selection of the transmission button 251 by the user.Furthermore, as shown in (b) in FIG. 8, the characters are displayed inthe character input field 150 displayed on the controlled apparatus 100.

Furthermore, as an example, a case is assumed in which the controlledapparatus 100 transmits to the control apparatus 200 operation typeinformation indicating that characters should be input character bycharacter, as information indicating the character string input scheme.In this case, as shown in (a) and (b) in FIG. 9, the control apparatus200 transmits the characters input using the English keyboard screen 262character by character, to the controlled apparatus 100.

Specifically, upon reception of the information, the network remotecontrol application 201 causes one character which is input to theEnglish keyboard screen 262 to be displayed in the display field 250 fora predetermined time period, and then transmitted to the controlledapparatus 100.

In other words, as shown in (a) in FIG. 9, the character “A”, which isdisplayed in the display field 250 in response to the operationperformed on the English keyboard screen 262, is transmitted to thecontrolled apparatus 100 without waiting for any operation by the user.As a result, as shown in (b) in FIG. 9, on the controlled apparatus 100,the character “A” is displayed in the character input field 150.

It is to be noted that the characters displayed in the display field 250may be updated (overwritten) with characters input to the controlapparatus 200 next, and may be displayed in parallel with the nextcharacters.

As described above, with the controlled apparatus 100 according to thepresent embodiment, operation type information indicating at least oneof the keyboard type and the character string input scheme for use inthe operation type identified by the second execution unit 110 isgenerated and transmitted to the control apparatus 200.

Here, the above-described character-by-character input is suitable forsearch string input to a search screen, for example. Specifically, asshown in FIG. 9, the controlled apparatus 100 realizes display of searchresult of incremental search in which the search result is updated on areceived character-by-character basis.

Furthermore, in the case of characters-by-characters input as shown inFIG. 8, for example, it is possible to transmit the characters after thecontrol apparatus 200 has completed kana-kanji conversion. Accordingly,the input scheme is suitable for the case where it is required to inputa long sentence to the controlled apparatus 100, such as when thecontrolled apparatus 100 is executing a mail application.

As described above, with the controlled apparatus 100 according to thepresent embodiment, the operation type of the controlled apparatus 100can be notified to the control apparatus 200, and the control apparatus200 can automatically display an operation screen suitable for thenotified operation type.

It is to be noted that when the controlled apparatus 100 is no longerwaiting for input of a character string (for example, when the browserapplication 102 is deactivated), the controlled apparatus 100 notifiesthe fact as the operation type information to the control apparatus 200.Thus, the control apparatus 200 can display the basic operation screen260 again.

As described above, with the network control system 10 according to theembodiment of the present invention, the user can operate the controlledapparatus 100 intuitively and easily using the control apparatus 200. Inother words, processing for the operation from the control apparatus 200to the controlled apparatus 100 is performed efficiently.

The network control system according to an aspect of the presentinvention has been described based on the embodiment. However, thepresent invention is not limited to the embodiment. Other forms in whichvarious modifications apparent to those skilled in the art are appliedto the embodiment, or forms structured by combining elements ofdifferent embodiments are included within the scope of the presentinvention, unless such changes and modifications depart from the scopeof the present invention.

For example, the network control system 10 in the present embodiment mayinclude a plurality of controlled apparatuses including the controlledapparatus 100. In other words, the network control system 10 may includea plurality of controlled apparatuses operable from the network controlsystem 10 and the control apparatus 200.

The control apparatus 200 can detect such controlled apparatuses by, forexample, broadcasting an apparatus detecting packet to the network 300.Furthermore, the control apparatus 200 may display, when detectingcontrolled apparatuses, a selection screen for allowing the user toselect one of the controlled apparatuses to be operated.

Furthermore, the controlled apparatuses do not have to be of the sametype. For example, if the DTV, HDD recorder, and BD player have afunction structure as the controlled apparatus 100 shown in FIG. 2, thecontrol apparatus 200 can control these apparatuses efficiently.

Furthermore, as an example of the operation screen to be displayed onthe control apparatus 200 when the controlled apparatus 100 is waitingfor input of a character string, the English keyboard screen 262 isshown in FIG. 6 and others.

However, a keyboard screen dedicated to input of languages other thanEnglish, such as Japanese, may be displayed on the control apparatus 200when the controlled apparatus 100 is waiting for input of a characterstring.

In this case, it is sufficient if information indicating a type oflanguage of the keyboard is included in the operation type informationtransmitted from the controlled apparatus 100. Furthermore, the controlapparatus 200 may generate and display a keyboard screen suitable for alanguage which can be handled by the controlled apparatus 100 indicatedin the capacity information transmitted from the controlled apparatus100.

INDUSTRIAL APPLICABILITY

With the present invention, users can operate digital home electronicapparatuses more easily. In other words, it is possible for a controlapparatus to efficiently operate a digital home electronic apparatuswhich is a controlled apparatus. Accordingly, the present invention isuseful as a network control system or the like for controlling digitalhome electronic apparatuses via a network.

REFERENCE SIGNS LIST

-   10 Network control system-   100 Controlled apparatus-   101 Broadcast viewing application-   102 Browser application-   110 Second execution unit-   111, 211 GUI control unit-   112, 212 Input control unit-   121, 221 Display unit-   122 IR Remote controller controlling unit-   123 Mouse control unit-   130 Operation type information generation unit-   131, 231 Protocol processing unit-   132 Second communication unit-   150, 151 Character input field-   152 Number input field-   200 Control apparatus-   201 Network remote control application-   210 First execution unit-   222 Touch-pad control unit-   230 Operational command generation unit-   232 First communication unit-   250 Display field-   251 Transmission button-   260 Basic operation screen-   261 Touch-pad mode screen-   261 a Scroll bar-   261 b Browser activation button-   262 English keyboard screen-   263 Number keyboard screen-   300 Network

The invention claimed is:
 1. A controlled apparatus which performsprocessing corresponding to an operational command transmitted from acontrol apparatus via a network, the controlled apparatus comprising: anexecution unit configured to execute an application program, andidentify an operation type which corresponds to processing performed bythe application program and is a type of operation accepted on anoperation screen output by the controlled apparatus; an operation typeinformation generation unit configured to generate operation typeinformation indicating the identified operation type; and acommunication unit configured to output the generated operation typeinformation to the network, wherein: the execution unit is configuredto, when a change in an operation type occurs, identify the operationtype after the change, the change in an operation type being caused by achange in the operation screen, the operation type informationgeneration unit is configured to generate operation type informationindicating the operation type after the change, the communication unitis configured to output the generated operation type informationindicating the operation type after the change to the network, theexecution unit is configured to, when the operation screen is changedfrom a state where a cursor is movable to a state where the cursor is ina character input field, identify a state of waiting for an input ofcharacter strings, as the operation type after the change, the operationtype information generation unit is configured to generate the operationtype information indicating the state of waiting for the input ofcharacter strings, the communication unit is configured to output theoperation type information indicating the state of waiting for the inputof character strings, to the network, the operation type informationgeneration unit is configured to generate the operation type informationincluding a keyboard type which is information indicating that thecontrolled apparatus is in a state of waiting for the input of anEnglish character string or in a state of waiting for the input of anumber character string, and the keyboard type is the identifiedoperation type.
 2. An apparatus control method for a controlledapparatus which performs processing corresponding to an operationalcommand transmitted from a control apparatus via a network, theapparatus control method comprising: executing an application program;identifying an operation type which corresponds to processing performedby the application program and is a type of operation accepted on anoperation screen output by the controlled apparatus; generatingoperation type information indicating the identified operation type; andoutputting the generated operation type information to the network,wherein: in the identifying, when a change in an operation type occurs,the operation type after the change is identified, the change in anoperation type being caused by a change in the operation screen, in thegenerating, operation type information indicating the operation typeafter the change is generated, in the outputting, the generatedoperation type information indicating the operation type after thechange is output to the network, in the identifying, when the operationscreen is changed from a state where a cursor is movable to a statewhere the cursor is in the character input field, a state of waiting foran input of character strings is identified as the operation type afterthe change, in the generating, the operation type information indicatingthe state of waiting for the input of character strings is generated, inthe outputting, the operation type information indicating the state ofwaiting for the input of character strings is output to the network, inthe generating, the operation type information including a keyboardtype, which is information indicating the controlled apparatus is in astate of waiting for the input of an English character string or in astate of waiting for the input of a number character string, and thekeyboard type is the identified operation type.
 3. A controlledapparatus which performs processing corresponding to an operationalcommand transmitted from a control apparatus via a network, thecontrolled apparatus comprising: an execution unit configured to executean application program, and identify an operation type which correspondsto processing performed by the application program and is a type ofoperation accepted on an operation screen output by the controlledapparatus; an operation type information generation unit configured togenerate operation type information indicating the identified operationtype; and a communication unit configured to output the generatedoperation type information to the network, wherein: the execution unitis configured to, when a change in an operation type occurs, identifythe operation type after the change, the change in an operation typebeing caused by a change in the operation screen, the operation typeinformation generation unit is configured to generate operation typeinformation indicating the operation type after the change, thecommunication unit is configured to output the generated operation typeinformation indicating the operation type after the change to thenetwork, the execution unit is configured to, when the operation screenis changed from a state where a cursor is movable to a state where thecursor is in a character input field, identify a state of waiting for aninput of character strings, as the operation type after the change, theoperation type information generation unit is configured to generate theoperation type information indicating the state of waiting for the inputof character strings, the communication unit is configured to output theoperation type information indicating the state of waiting for the inputof character strings, to the network, the operation type informationgeneration unit is further configured to generate capacity informationidentifying one of (i) a language outputtable or interpretable by thecontrolled apparatus and (ii) information indicating an applicationprogram executable by the execution unit, and the communication unit isconfigured to output the operation type information and the capacityinformation to the network.